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C , Abstract. We study the performances of stochastic heuristic search algorithms on Uniquely 

Extendible Constraint Satisfaction Problems with random inputs. We show that, for any 
Cfj ' heuristic preserving the Poissonian nature of the underlying instance, the (heuristic-dependent) 

largest ratio a a of constraints per variables for which a search algorithm is likely to find solutions 
ON ' is smaller than the critical ratio ay above which solutions are clustered and highly correlated. 

' In addition we show that the clustering ratio can be reached when the number k of variables 

per constraints goes to infinity by the so-called Generalized Unit Clause heuristic. 

X. 

1. Introduction 

The application of statistical mechanics ideas and tools to random optimization problems, 
initiated in the mid-eighties [1], has benefited from a renewed interest from the discovery of 
phase transitions in Constraint Satisfaction Problems (CSP) fifteen years ago. Briefly speaking, 
one wants to decide whether a set of randomly drawn constraints over a set of variables admits (at 
least) one solution. When the number of variables goes to infinity at fixed ratio a of constraints 
per variable the answer abruptly changes from (almost surely) Yes to No when the ratio crosses 
some critical value a s . Statistical physics studies have pointed out the existence of another 
phase transition in the Yes region [2, 3]. The set of solutions goes from being connected to a 
collection of disconnected clusters at some ratio < a s , a translation in optimization terms of 
the replica symmetry breaking transition identified by Parisi in mean-field spin glass theory. 

It is expected that this clustering transition may have dynamical consequences. As replica 
symmetry breaking signals a loss of ergodicity, sampling algorithms (e.g. Monte Carlo procedure) 
run into problems at that transition. A quantitative study of the slowing down of MC scheme was 
done in [4] for the case of the /c-XORSAT model where constraints are simply linear equations 
(modulo 2) over k Boolean variables (for an introduction, see [5] and references therein). Yet, 
finding a solution should in principle be easier than sampling, and the exact nature of the 
relationship between the performances of resolution algorithms and the static phase transitions 
characterizing the solution space is far from being obvious [6]. The present paper is a modest 



step in elucidating this question for the /c-XORSAT problem, and some related NP-complete 
problems sharing the same random structure. 

Hereafter we consider simple stochastic search heuristic algorithms working in polynomial 
(linear) time for solving /c-XORSAT instances [8, 5]. By successively assigning variables 
according to some heuristic rules those algorithms either produce a solution, or end up with 
a contradiction. The probability that a solution is found is a decreasing function of the ratio a, 
and vanishes above some heuristic-dependent ratio a a in the infinite size limit. We show that 
"a < 014 for any assignment heuristic in the class of rules preserving the Poissonian structure of 
the instance. In addition, we determine the most efficient heuristic, that is, the one maximizing 
a a in this class and show that for large k, the two critical ratios match, a a {k) ~ oi^ik) ~ log k/k. 

The plan of the paper is as follows. In section 2 we define the random /c-XORSAT decision 
problem and its extension, as well as the search algorithms studied. Section 3 presents a method 
to characterize the phase diagrams of those random decision problems, depending on the content 
(numbers of constraints over j variables, with j ranging from 1 to k) of their instances. We 
show that all important information is encoded in a unique 'thermodynamical' potential for the 
fraction of frozen variables (backbone) . The analysis of the dynamical evolution of the instance 
content is exposed in section 4. These dynamical results are combined with the static phase 
diagram in section 5 to show that the success-to-failure critical ratio of search heuristic, a a , is 
smaller than the ratio corresponding to the onset of clustering and large backbones, a^. We 
then show that the so-called Generalized Unit Clause heuristic rule is optimal (in the class of 
Poissonian heuristics) and its critical ratio a a is asymptotically equal to ad in the large k limit. 
Our results are discussed in section 6. 

2. Definitions 

2.1. Decision problems 

The decision problems we consider in this paper are (k, <i)-Uniquely Extendible (UE) Constraint 
Satisfaction Problems (CSP) defined as follows [7]. One considers N variables xi € {0, 1, ■ ■ ■ , d — 
1}. A UE constraint, or clause, is a constraint on k variables such that, if one fixes a subset of 
k — 1 variables, the value of the fc-th variable is uniquely determined. A (k, d)-UE-CSP formula 
is a collection of M = aN clauses, each involving k variables (out of the N available ones). A 
solution is an assignment of the N variables such that all the clauses are satisfied. /c-XORSAT 
corresponds to d = 2 and is solvable in polynomial time with standard linear algebra techniques. 
For d = 3 the problem is still in P, while for d > 4 it has been shown that (3, d)-UE-CSP is 
NP-complete [7]. 

A random formula is obtained by choosing, for each clause, the k variables, and the actual 
UE constraint, uniformly at random. It is known that, in the infinite size limit N — > oo and at 
fixed clause-to-variable ratio a, [7, 11, 12, 13]: 

• there is a critical ratio a s (k) such that a random (k, d)-UE-CSP is almost surely satisfiable 
(respectively, unsatisfiable) if a < a s (k) (respectively, a > a s (k)). 

• in the satisfiable phase there is another phase transition at some ratio a^(/c) such that: 

- for a < oid{k) the space of solutions is 'connected': with high probability there is 
a path in the set of solutions joining any two solutions such that a step along the path 
requires to change 0(1) variables. 

- for a > dd{k) the space of solution is disconnected into an exponentially large number 
of clusters, each one enjoying the above connectedness property, and far away from each 
other (going from one solution in one cluster to another solution in another cluster requires 
to change O(N) variables). In addition, in each cluster, a finite fraction of variables are 
frozen i.e. take the same value in all solutions (backbone). 



2.2. Search algorithms 

We will consider simple algorithms acting on the formula in an attempt to find solutions. Those 
algorithms were introduced and analyzed by Chao and Franco [8] (see [9] for a review). Briefly 
speaking, starting from a randomly drawn formula, the algorithm assigns one variable at each 
time step according to the following principles: 

• If there is (at least) one clause of length one (called unit-clause) then satisfy it by adequately 
assigning its variable. This rule is called unit propagation. 

• If all clauses have length two or more, then choose a variable according to some heuristic 
rules. Two simple rules are: 

- Unit Clause (UC): pick up uniformly at random any variable and set it to a random 
uniform value in {0, • ■ ■ , d — 1}; 

- Generalized Unit Clause (GUC): pick up uniformly at random one of the shortest 
clauses, then a variable it this clause, and finally its value. 

In this analysis, we will discuss a general heuristics in which the variable to be set is chosen among 
those that appear in the clauses of length j with some probability Pj(C\, • • • , C&), depending in 
general on the number of clauses of length j present in the formula, that we shall call Cj . Unit 
propagation implies that if C\ ^ 0, then pj = 5j t \. We consider also the possibility that the 
variable is chosen irrespective of the clause length, then Y$=iPj ^ 1- 

Both UC and GUC are special cases of this general class: in UC variables are chosen at 
random, irrespectively of the clauses they appear in (if any), so that pj = unless there are 
unit clauses; GUC corresponds to pj = 5jj* where j* is the length of the shortest clause in the 
system. Notice that since the variables are selected independently of their number of occurrences, 
the latter remains Poissonian under the action of the algorithm (even though the value of the 
parameter in the distribution of occurrences may vary). More involved heuristics do exist but 
will not be analyzed here. 

Under the action of the algorithm clauses get reduced (decrease in length) until they disappear 
once satisfied. The algorithm stops either when all clauses have been satisfied or when two 
incompatible unit-clauses have been generated e.g. x = and x = 1. In the latter case the 
algorithm outputs 'I do not know whether there is a solution', while in the former case the 
output reads 'Satisfiable' and returns a solution to the formula. The probability of success, 
that is, the probability (over the choices of the algorithms and the formula) of getting the 
'Satisfiable' output vanishes above some heuristic-dependent ratio a a (< a s ) in the infinite N 
limit. This success-to-failure transition coincides with the polynomial-to-exponential transition 
of backtracking algorithms [5, 10]. 

3. 'Thermodynamical' Characterization of the Space of Solutions 

Under the action of the algorithm the length of the clauses changes; therefore the initial (k, d)- 
UE-CSP formula where all clauses have length k evolves into a formula with some distribution 
of clauses of different lengths. We wish then to characterize the space of solutions of a generic 
d-UE-CSP formula made by N variables and by {C°} J= 2,...,fe clauses of length j, assuming that 
there are no unit clauses. This characterization will be useful to analyze the performance of 
search algorithm in the following. 

3. 1 . Leaf removal procedure and its analysis 

Our starting observation is that, due to the UE property, when a variable has a unique 
occurrence in the formula, then the clause it appears in can always be satisfied. Hence the 
subformula obtained by removing this clause is equivalent (in terms of satisfiability) to the 
original system [11]. The interest of this remark is that it can be iterated, and more and more 



clauses eliminated. Monitoring the evolution of the formula under this procedure, called leaf 
removal, provides us with useful information on the nature of the solution space [12, 13, 14]. 

One clause is removed at each time step. After T steps we denote by Cj(T) the number of 
clauses of length j. Those numbers obey the evolution equations (in expectation), 



C J (T + l)-C J (T) = 



j Cj{T) 



£*, =2 j'C>(T) 



(1) 



where the denominator is the total number of occurrences of all variables appearing in the 
formula. The r.h.s. of (1) is simply (minus) the probability that the unique-occurrence variable 
is drawn from a clause of length j. 

In addition let us define the number N(_(T) of variables appearing in £ equations exactly. The 
evolution equations for those numbers are (in expectation) 



N e (T + 1) - N £ (T) = ]T 



j(j-l)C 3 {T) 



(£ + l)N e+1 (T)-£N £ (T) 



(2) 



The above is easy to interpret. The second term in the square bracket on the r.h.s. is the 
average number of removed variables (other than the single-occurrence variable), that is, the 
average length of the removed clause minus one. The first term expresses that, if one of those 
variables appeared £ + 1 times before its removal, the number of its occurrences has decreased 
down to £ after the removal. Finally, the two 5 correspond to the elimination from the system 
of the single-occurrence variable. 

In the large N limit we may turn those finite difference equations over extensive quantities 
Cj,N£ into differential equations for their intensive counterparts Cj = Cj/N,ri£ = Ng/N as 
functions of the reduced number of steps, r = T/N. The outcome is 
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dr 
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~dV 



where J\f(r) = J2j=2J c j( T ) 
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(j = 2,...,k), 
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(4) 



J2e>i ^ n i( T )- The initial conditions are 
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(5) 



where Ao is determined by J2e^ n e(ty = Ao = J2j i c i(0)- 

It is easy to check that equations (3) are solved by Cj(r) 
It is convenient to introduce the generating function 



Cj(0) b(r) j provided f § = -1. 



G(&) = 5>(0)6>'. 

j=2 



(6) 



Derivative(s) of G with respect to its argument will be denoted by prime(s). We have that 
M{t) = 6(t)G'(6(t)). In addition, we define 7 (r) = EjCj(t) = G(6(t)). We deduce the 
equation for b(r): 



dq_ _ Mdb 
dr b dr 



r = 7 (0)- 7 (r) = ^ Cj (0)(l-6(r) 
i=2 



(7) 



The interpretation of the equation above is just that at each step of the leaf removal one equation 
is eliminated. 

The solution to (4) remains Poissonian at all times for all I > 2. Substituting n^(r) = 
g-A(-r) Hj) we obtain an equation for A(r): 

G"{b) 



dX E,> 2 i(j - !)cj( r ) 



dr (E 7 >2JCi(^)) 2 



G'{bf 



A(r) , (8) 

b=b(r) 



A(r) = - 

with the initial condition imposed by A(0) = Ao = J2jJ c j(fi) = C'(l). From (7) we get 

dX dXdr G"{b) 



*L = -G'(b) so that 



-A , (9) 



db dr db G'{b) 
which is solved by 

A(6) = G'(b) , (10) 

where the normalization is fixed by the initial condition for A. (7) and (10) determine 6(r) and 
A(r), which describe the evolution of the formula under the action of the leaf removal algorithm. 

3.2. Static Phase Transitions 

The structure of the subformula remaining at the end of the leaf-removal (if any) is indicative 
of the nature of the phase corresponding to typical formulas, uniformly drawn at fixed {Cj}. 
Three phases are possible: the unclustered phase where formulas are satisfiable and the solutions 
form a unique cluster; the clustered phase where solutions are divided into many clusters; and 
the unsat phase where the typical formula is not satisfiable 

(i) Clustering transition: The leaf removal algorithm starts from 6=1, then 6 decreases 
according to (7) and the algorithm stops at the largest value of b such that n\ = 0, i.e. 
there are no more variables with unique occurrence. We have 

ni = E^-E^ = ^'(6)-E^- A(6) ^ 

= b\(b) - e~ A(6) A(6) [e A ^ - l] = A(6) [b - 1 + e~ x ^] , 

therefore 

ni = e> l-b = e- x{b) = e~ G '^ . (11) 

This equation always has the solution 6 = 0, that gives Cj = for all j when the algorithm 
stops. This corresponds to a backbone-free formula whose solution space is connected. On 
the other hand, if this equation admits non-trivial solutions 6 > 0, the algorithm stops when 
6 is equal to the largest of them, i.e. it is unable to eliminate all clauses in the formula. 
Then the space is clustered and the largest solution represents the fraction of variables in 
the backbone of each cluster [12, 13]. 

In the pure (k, d)-UE-CSP case, i.e. when c° = adj^, the critical ratio at which clustering 
appears decreases with k, from a^(3) ~ 0.818 to ad{k) ~ logk/k at large k. 

(ii) Sat/unsat transition: The formula is satisfiable when the subformula left by the removal 
algorithm has a solution. This happens with high probability if and only if the number of 
equations, given by G(b), is smaller than the number of variables, J2e>2 n £ [12, 13]. Using 
the condition n\ = 0, the satisfiability condition is 

G(b) <6+(l-6)log(l-6) . (12) 

For (k, (i)-UE-CSP, the critical ratio at which formulas go from typically satisfiable to 
typically unsatisfiable increases with k, from a s (3) ~ 0.918 to ctd(k) — > 1 at large k. 



3. 3. The potential for the backbone 

The outcome of the previous section can be summarized as follows. We considered a formula 
specified by a set {Cj}j=2,...,fc, or equivalently by the generating function (6). In the following 
we will drop the superscript to simplify the notation. We define the potential 

V(b) = -G(6) + 6+(l-6)log(l-6) . (13) 

The condition n\ = (11), is equivalent to V'(b) = 0. Thus, if V(b) has a single minimum in 
b = 0, the solution space is not clustered, while if there is another minimum at b ^ 0, there 
are clusters. Moreover, the condition for satisfiability (12), is that at the secondary minimum 
V(b) > 0. Examples are given in figure 2. 

The sat/unsat surface S s , that separates the sat and the unsat phase, is defined by the 
condition: 

S s = { Cj : V(b) = and V'(b) = admit a solution b > 0} . (14) 

The clustering surface S^, that separates the clustered and unclustered regions, is defined 
similarly by 

S d = { Cj : V'(b) = and V"{b) = admit a solution b > 0} . (15) 

The equations above have to be interpreted as coupled equations for (b,Cj); therefore S S ,S^ 
have dimension k — 2 and are surfaces in the space {cj}j=2,-,k of dimension k — 1. Note that 
in (14) and (15), one must always choose the largest solution for b, to which we will refer as b s 
and bd, respectively. 

In addition to the previous sets, in the following a special role will be played by the condition 
2c2 = 1, or equivalently V"(0) = 0, that defines the contradiction surface T, q : 

E, = { Cj : V»(0) = 0} . (16) 

The surface S g is simply a hyperplane of dimension k — 2. 

3.4- The phase diagram 

We draw a phase diagram in the space of the Cj by representing surfaces S s , S^, T, q . We focus 
on the region Cj € [0,1] for j = 3,...,k and C2 € [0,1/2]. Indeed, if one of the Cj > 1, the 
system is surely in the unsat phase [7] while if C2 > 1/2 the algorithm discussed above find a 
contradiction with very high probability. 

Examples of the phase diagram are in figure 1 for k = 3 and k = 4. There are some special 
"lines" (i.e. intersections of surfaces) on which we will concentrate. 

(i) Recall that S g is defined by V"(0) = and note that V'(0) = for all b, cj. Thus, on 
Tig, the point b = is a solution of both equations (14) and (15). The surfaces S s , are 
defined by the existence of solutions with b > 0, but they might intersect S g if for some 
values of {cj} the solution with b > merges with the solution 6 = 0. This happen when 
V"'(0) = 0, as this is the limiting case in which a saddle at b = bd > and a secondary 
minimum at b = b s > can merge for bd, b s — > 0. The condition V"'(0) = is equivalent to 
C3 = 1/6, and this defines the k — 3-dimensional surface 

S cr ii = {cj : c 2 = 1/2, c 3 = 1/6} , (17) 

to which we will refer as critical surface. It is easy to see that the three surfaces S s , E^, T, q 
are tangent to each other on the region of the critical surface where they intersect. To 
show that one must consider a displacement C3 = 1/6 + e and show that (15), (14) admit a 
solution with b s , bd ~ e if C2 — 1/2 ~ e 2 . We say that in this case the phase transitions are 
of second order because the order parameter b vanishes continuously at the transition. 



Figure 1. (Left) Schematic phase diagram of A;=4-UE-CSP. The full (black) curve is the surface 
Ed, the dot-dashed (red) surface is E s . The two surfaces meet along a portion of the line E cr jj, 
defined by C2 = 1/2 and C3 = 1/6 and represented as a dashed (blue) line. (Right, top and 
bottom) The sections of E^ (full, black) and of E s (dot-dashed, red), at fixed C2 (= 0, 0.1, 0.2, 
0.3, 0.4, 0.5 from top to bottom) as a function of C3 on the top panel, and at fixed C4 (= 0, 0.1, 
0.2, 0.3, 0.4, 0.5, 0.6, 0.7 from top to bottom) as a function of C2 in the bottom one. The lines 
corresponding to C4 = also represent the phase diagram of 3-UE-CSP. 

(ii) There is no a priori reason for which the three surfaces must cross at E cr jj. In fact, the 
solutions at b > might also disappear discontinuously, like in figure 2, and the surfaces E s 
and Ed can intersect the surface E 9 in regions different from E cr j£. This does not happen 
for k = 3 but happens for k = 4 for large C4, see figure 1. In this case the transition is called 
first order because the order parameter jumps at the transition. 

The generic phase diagram for all k has the shape of the one for k = 4 which we report in 
figure 1, left panel. 

4. Search Trajectories in the Space of Formulas 

The heuristics we defined in section 2 enjoy the property that, after any number of steps of 
the algorithm, the reduced formula is uniformly distributed over the set of remaining N — T 
variables conditioned to the numbers Cj(T) of clauses of length j (= 2,...,k) [8, 9]. This 
statistical property, combined with the concentration phenomenon taking place in the large N 
limit, allows us to study the evolution of the average clauses densities Cj(t) = Cj(T)/N on the 
time scale t = T/N (fraction of assigned variables), which defines a trajectory in the Cj's space. 
Note that these Cj(t) are defined with respect to N, therefore the actual clause density for the 
reduced system of N — T variables are Cj(t) = c,-(i)/(l — t). The trajectory of the c~j(t) moves 
in the Cj space of the previous section 1 . 

Initially we have Cj(0) = a 5jk, i.e. the evolution starts on the axis at c/% = a. The 



1 The reader should keep in mind this change of notation to avoid confusion in the following arguments 



evolution equation for the densities take the form of first order differential equations, 



(j + l)c i+ i - j Cj 
l-t 



- Pj (t) . 



(18) 



The interpretation of the equations above is the following. Let us consider an interval [t, t + dt] 
of continuous time that corresponds to AT ~ Ndt time steps of the algorithm. The first term 
on the r.h.s. arises from the decrease by one of the length of the clauses that contained the 
variable just assigned by the algorithms during this interval. The second term corresponds to 
an additional loss of clauses which is present when the variable is selected from a clause of 
length j: as the heuristics explicitly chooses an equation (and a variable therein) of length j 
with probability pj (see section 2), this equation will be reduced irrespectively of the number of 
other occurrences of the variable. Hence Pj(t) is given, for j > 1, by 



where both pj,pj + i depend on their arguments (numbers of clauses) and (•) represents the 
average over AT defined in (19). Here Pk+i = 0. Note that the case j = 1 is special as all 
clauses of length one that are produced are immediately eliminated. On average 



clauses of length 2 become of length 1 and are then eliminated by unit propagation. The total 
fraction of eliminated clauses is 



where the last inequality follows from (19). As only clauses of length one are eliminated, the 
violation of (21) can only happen if too many such clauses are generated. This corresponds 
to pi — > 1 _ ; in this case a contradiction occurs with high probability and the algorithm stops 
with the 'Don't know' output. When p\ — > 1 _ , the algorithm makes only unit propagations and 
pj — > + for all j > 2. For this reason we called the plane pi = 1, i.e. c<i = 1/2, contradiction 
surface. 

4.1. Unit Clause (UC) 

In the UC heuristic variables are chosen at random when there is no unit clause. Hence pj = 
for j = 2, • • • , k. The solution to (18) is Cj(t) = a(^)(l - t)H k ~ 3 . The algorithm will generate 
a contradiction with high probability (w.h.p.) if the average number of unit clauses starts to 
build-up, i.e. if 2c2(t)/(l — t) > 1. This gives an equation for the value of a at which the 
probability that the algorithm finds a solution vanishes: for k = 3, ai UC ^ = 2/3. 

4-2. Generalized Unit Clause (GUC) 

In the GUC heuristic the algorithm always fixes a variable appearing in the shortest clauses. In 
the continuum limit Cj = for j smaller than a given value; therefore we define 




(19) 




(21) 



f(t)=mm{j: Cj (t)>0} , 



(22) 



the minimal length of clauses with positive densities. We also define 



t*(j) = mm[t : Cj _i(t) > 0] (23) 

the time at which j* jumps down from j to j — 1. Essentially, the algorithm picks one clause of 
length j* and assigns successively all the variables in this clause until the clause disappears. But 
in doing so, other clauses of length j < j* are generated and have to be eliminated to recover 
the situation in which Cj = for all j < j*; for this reason pj* is not given exactly by l/j*. 
When the number of generated clauses is so high that the algorithm is unable to remove them, 
Cj*-i becomes different from and j* jumps down by 1. The resulting motion equations for the 
clause densities are, for j > j*(t): 

■ r ( .,_ (j + l)c 3+ i(t)-jc 3 (t) fl (j-l) Cj (t) \ 



The transition times t* are given by 

c 3 (t*(j)) 



l-t "j(i-l)' 



(25) 



where the algorithm is no more able to remove the clauses of length j* because too many clauses 
of length j* — 1 are being generated by propagations. 

Comparing with (18) above, we observe that in the interval t <G [t*(j + 1), t*(j)], where j* = j, 
only two pj are different from 0: 

_ 1 (j*-l)c 3 -(t) _ fc r (t) 

Pi* -j* — t — ' p ^- x ~ T^F ' (26) 

the first representing clauses of length j* which are directly eliminated, the second representing 
the clauses of length j* — 1 that are produced and subsequently eliminated in the process. In 
this interval of time, the ratio Cj*(t)/(1 — t) increases from to — 1) from condition 

(25). Then 

W)- m = {Pr+Pr - l) -m^^ (27) 

which is consistent with (but stronger than) (21) above. 
5. Analysis of the "dynamic" phase diagram 

Consider now a given heuristic, and a generic (k, d)-UE-CSP formula specified by its clause-to- 
variable ratio a. The formula, in the Cj space, starts on the axis at Ck = a. The evolution 
of the formula under the action of the algorithm is represented by a trajectory {cj(t, cx)}j=2,—,k 
or equivalently by G(b;t,a) = Ylj=2 &Cj(t, a), that depends on a through the initial condition 
G(b;0, a) = ab k . We define a potential V(b;t, a) by replacing in (13) G(b) — > G(b;t,a)/(1 — t); 
the normalization (1 — t) is due to the fact that the Cj = Cj/N are divided by N instead of 
N-T. 

We follow the evolution of the formula by looking at the times at which the trajectory 
starting at = a at time crosses the surfaces S s , S^, T, q defined in section 3.3, which we call 
t s (a),td(a),tq(a) respectively. As an example, in figure 2 we report the potential at different 

times during the evolution of a formula according to the UC heuristic for a > afa^ . 

We draw a "dynamic phase diagram" by representing in the (t, a) plane the lines separating 
the unclustered, clustered, unsat and contradiction phases, which we call ad(t),a s (t),a q (t) and 



0.2 



V(b) 




0.2 0.4 , 0.6 0.8 



Figure 2. An example of the potential V(b; t, a) plotted (from top to bottom) at times 
t = {0,t d = 0.02957, 0.07327, t s = 0.11697,0.20642} during the evolution of a (3, d)-UE-CSP 
formula with a = 0.8 under the UC heuristic. In the unclustered region it is a convex function 
of b with a global minimum in b = 0. On the clustering line td it first develops a secondary 
minimum. On the sat/unsat line the value of V at the secondary minimum becomes equal to 0. 



are just the inverse of the times defined above. Examples in the case of the UC and GUC 
heuristics are given in figure 3. 

From the general properties of the function V(b; t, a) we can deduce a number of properties 
of the lines a d (t), ct s (t), a q (t). We will show that the three lines intersect at a "critical point" 
(t a , a a ), located at a a < a d , under the more general conditions. This implies that the algorithm 
stops working at the value a a < ad, which is our central result: Poissonian search algorithm 
cannot find a solution in polynomial time in the clustered region. 



5. 1 . Equations for the transition lines 

The generating function G(b; t, a) satisfy an evolution equation which is easily derived from (18): 



G(b;t,a) 



L^G'(b;t,a)-F(b;t,a) 
2c 2 (t) i 



F(b;t,a) ee -^l b + J2p 3 (tW =J2Pj(^ 

j=2 j=l 



(28) 
(29) 



Performing the total derivative with respect to t of the first condition (V = 0) in (15) for 
{otdi bd) and using the second condition, V" = 0, we have 



da dt 

Using the definition (13) we have 
V'(b;t,a) = 



dV da d ■ , dad 

dt 



1-t 



G'(b;t,a) + 



V'(b d ;t,a d ) 
& d ;t,a d ) 



G'(b;t,a) 



1-t 



(30) 



(31) 




Figure 3. (Left) Phase boundary lines in the (t, a) plane for the UC and GUC heuristics for 
k = 3. The three lines meet at the critical point (t a ,a a ) at which the algorithm is no more 
able to find a solution (black dot). (Right) The generic shape of the clustering and of the 
sat/unsat lines. The possibility of a maximum cannot be excluded, but in any case t must be a 
single-valued function of a, meaning that if the algorithm enters the cluster (or unsat) phase it 
cannot escape at later times. 



Then 



dV 
da 



dad 
dt 



1 dG' 1 v-^ dcAt, a) 

— = y jtjr> 1 — J - 

1-tda 



i>2 



da 



G'(b;t,a) + ^-; 



G'(b;t,a) 



d a G'(b;t,a) 

Using (28) and differentiating with respect to b we have 



(32) 



(33) 



a=a d {t),b=b d (t) 



G'Ur.l.a) + G 'fo*' a ) = LJg"(6; t, a) - F'(b; t, a) . 



Now using V"(b; t, a) 
and finally we get 



l-i 

dad 
dt 



(34) 

+ and V"(b d , t) = we have G"Q>; t,a) = l for b = b d 

(35) 



l-i l-i 

G"(b;t,a) . 1 ^ AX rll(u. , ; . n 1 



1-F'(b;t,a) 



d a G'(b;t,a) 



a=a d (t),b=b d (t) 



A very similar reasoning leads to the following equation for the sat/unsat line: 



da s 
dt 



b - F(b;t,a) 



d a G(b; t, a) 



a=a s (t),b=b s (t) 



(36) 



The equation for the contradiction line is easily derived from its definition c%{t, a) = C2 ^_'°) 



2- 



which immediately gives 



da q 
~~dt 



l + 2c 2 (i,a) 



2<9 Q c 2 (i, a) 



a=a q (t) 



(37) 



5. 2. General properties of the transition lines 

We wish to show that the transition lines td(a),t s (a) and t q {a) in the (cx,t) plane are single- 
valued functions of a, and that they meet in a point (a a ,t a ) where they have infinite slope and 
are therefore tangent to each other; the value a a correspond to a trajectory which is tangent to 
the crytical surface Scrit- 

Our argument goes as follows: 

(i) We defined a a as the value of a for which the probability of finding a solution for the 
chosen heuristic vanishes. Then the trajectory 2 corresponding to any a > a a must cross 
the contradiction surface, while the trajectory corresponding to any a < a a must not cross 
it, so that the trajectory corresponding to a a must be tangent to the contradiction surface 
Ti q . The latter trajectory is tangent to Ti q when c 2 (t) = 1/2, ^c 2 (t) = 0; the solution to 
these conditions gives t a and a a . 

Moreover, c 2 (t) = 1/2 implies that pi = 1 which then implies pj = for all j > 2, as already 
discussed. Then we have 



d i > 



d 2c 2 (t) 



2c 2 (t) 2c 2 (t) 



= 



c 2 (*) = - 



C2(t) 



1 



dt 1 - t 1-t ' (1 - t) 2 
which, together with the equations of motion (18) and p 2 = gives 

c 2 (t) dc 2 (t) 3c 3 (t) - 2c 2 (t) ... cg(t) 1 c 2 (t) 



1-t 



dt 



1-t 



1-t 3 1 -t 



(38) 



(39) 



Therefore the point where the trajectory for a = a a is tangent to the contradiction surface 
belongs to the critical surface T, crit . From equation (37) it is clear that since c 2 = —1/2, 
the function t q (a) has infinite slope in (t a , a a ), as in figure 3. 

(ii) Next we show that the numerators of the fractions appearing in d^(i) and a s {t) are strictly 
positive if t < t q (a), i.e. in before a contradiction is found. Using the definition (19) we 
can write: 



(pi> + 5>"- 2 (&-1)<p;> 



(40) 



F(b;t,a) = '£Pj( t ) bj = b 

i=i L i= 2 

k k 

F'(b;t,a) = Y,jP ] itW- 1 = {p 1 ) + Y,^ 2 ^-j^-b)\{p ] ) ■ 

The coefficients in front of (pj) > in the sums above are always smaller than 1, 
independently of j, so that 



F(b;t,a) < b 



(Pi) + E (Pi) 

3=2 



<b, 



F'(b;t,a) < (pi) + '£(p j )<l. 

3=2 



(41) 
(42) 



The functions F(b;t,a) and F'(b;t,a) are to be computed in b = b s (t,a) or b = bd(t,a) 
in equations (35) and (36). Both b s and b& are strictly smaller than 1 for all (t,a), as one 
can directly show from their definitions because V'(b — ► 1) — > oo. Then the coefficients in 
the sums in (40) are strictly smaller than 1, and the only solution to F = b or F' = 1 is 
(pj) = S±j, which happens only on the contradiction line. 



2 Recall that we are here talking about average trajectories. 



(iii) The denominators in equations (35), (36) are surely positive at t = 0, as G(b;0, a) = ab k 
independently of the heuristic. If they remain positive at all times, then ad(t),a s (t) < at 
all times, or equivalently ^ < at all a, so that td, t s always increase on decreasing a. 
The other possibility is that the denominator in (35) crosses zero and become negative, 
leading to a maximum in td(a), which will then decrease on decreasing a. Possibly the 
denominators can vanish again, giving rise to a sequence of maxima and minima, see right 
panel of figure 3. 

What is important is that the numerator is always strictly positive, and as a consequence 
td(a) or t s (a) are single-valued functions of a. In fact, for td(a) or t s (a) to be multiple- 
valued functions of a, at some point their slope must become infinite, which is excluded by 
the analysis above. 

(iv) The statement above, that td(a) and t s (a) are single valued functions of a, implies that 
if a trajectory enters the clustered or unsat phase, it cannot exit from it. This is enough 
to show that a a < ad', in fact, the trajectory for a = a a cannot start inside the clustered 
phase, as it would not be able to escape and reach the origin, which is required to find a 
solution. 

(v) In general the function c^(t) increases until it reaches a maximum and then decreases 
to 0. For a = a a the value at the maximum is Ci = 1/2. For a > a a , the value 
at the maximum is C2 > 1/2, therefore the contradiction C2 = 1/2 is reached before 
the maximum, when C2 is still increasing. Then > at the contradiction point. 
Performing a simple computation similar to equations (38), (39), one can show that the 
trajectories for a > a a meet the contradiction surface at C3 > 1/6. Notice then that, 
as it is evident in figure 1, the trajectories corresponding to a > a a must enter first the 
clustered and then the unsat phases in order to reach the contradiction surface, therefore 
for a < a a one has td{p) < t s (a) < t q (a). On the contrary the trajectories corresponding 
to a < a a must stay away from the clustering and sat/unsat surfaces, otherwise they could 
not exit and should meet the contradiction surface: therefore for any a < a a , td(a) and 
t s (a) do not exist. For a — > a+, as the surfaces Sd,S s ,S g are tangent in S cr jt, one has 
td( a a) = t s {a a ) = t q {a a ) = t a and the three curves have infinite slope as all the numerators 
in equations (35), (36), (37) vanish on the contradiction surface. This is indeed what is 
observed in figure 3 for the UC and GUC heuristics, and this argument confirms that this 
is the generic behavior for all the heuristics in the class considered here. 

This structure is particularly evident for UC, where 

G^ c \b;t,a) =a[l-(l -b)(l-t)] k - at k ~ l [kb{\ -t) + t] . (43) 

From (43) it is straightforward to check that d a G(b;t,a) > 0, d a G'(b;t,a) > 0, if b > 0. Then, 
as F(b;t,a) = 2b ^^ for UC, both atd(t) and a s (t) are proportional to — 1. This means 

that a s ,a.d are decreasing functions of t below the contradiction line. 

The conclusion is that for a generic Poissonian heuristic, the three lines cross at a critical 
point (t a ,a a ) which depends on the heuristic. Above a a the heuristic will cross all the lines 
and find a contradiction. From the properties of the dynamical line, we have that generically 
ct a < ctd, that is no Poissonian search heuristic can find a solution in polynomial time above 
ad, as stated at the beginning of this section. The natural question is then if there exists an 
heuristic that saturates the bound, i.e. such that a a = ad- From the discussion above it is clear 
that this is possible only if c\d{t) = 0, i.e. the dynamical line in the (t, a) plane is a straight 
vertical line, which is possible only if the numerator in (35) is identically vanishing. 



5.3. Optimality of GUC 

It is quite easy to see that GUC is the heuristic that locally optimizes the numerator in (35). 
Indeed, from the definition F'(b;t,a) = Y^]=\3^ 1 Pj an d the bound F'(l,t) < 1, it is clear 
that F'(b; t, a) is maximized by maximizing pj for the smallest possible j, i.e. by picking clauses 
from the shortest possible ones, that is GUC. Unfortunately a general proof of the optimality 
of GUC for finite k seems difficult, because one should prove that GUC optimizes globally the 
clustering line, and also control the denominator in (35). In this section we will show that for 
k — > oo, GUC is optimal in the sense that = and = a a at leading order in k. 

From the definition j(t) = — X^ =2 c./(i) and integrating over time the bound (27), we have 
for GUC: 

f* dt' ,. ft alt' 

a - / — — - < -7 (t) < a - / — — . (44) 

Jo j*(t')-l ~ rw - Jo j*(t>) v ; 

or, equivalently, 

„ _ £ fd) -ev + 1) < _ 7(t) < a _ £ m - m + 1) (45) 

j j 

where the sums are limited to the values of j that are reached during the search. In the large k 
limit, provided the hypothesis 

t*U)-t*(j + l) = l + o(l/k) (46) 



holds for most j, we obtain 

k 

-7(t)^«-rE7- ( 47 ) 

*;•(*) J 

The hypothesis (46) is well supported by numerical data, as shown in figure 4. As the sum of the 
inverse of the first k integers is equivalent to log k (harmonic number) we see that the minimal 
value of j* over t is much larger than 2 if a is much smaller than log kj k. Therefore 

Qa >^. (48) 
k 

The r.h.s. of the above inequality coincides with the asymptotic scaling of the clustering critical 
ratio (section 3.2). Since the results of the previous section require that a a < ad, we obtain 

that a^ UCS> = ad — logk/k at the leading order in k — > oo. As a comparison, it is easy to see 
that for UC the threshold for large k is cta UC ^ — e/k, which is therefore much lower than the 
threshold for GUC. 

These arguments are supported by numerical simulations that we performed up to k = 2 16 , 

in which the equations of motion (24) are integrated as finite differences equations for all values 

( cue) 

of j (see figure 4). The numerical investigation confirms that ka a is very well fitted by 
log A; + 2.15 for k in the range 2 8 -j- 2 16 . Moreover, a finite size scaling analysis (with respect to 
k) of the data shown in figure 4 shows that 

k[t*(j)-t*(j + l)] = l + k» xf(j/k) (49) 



where f(x) is a function independent on k which behaves as for x close to 0. From the 
numerical data, it appears that v = [i = 0.5, which confirms that the first correction to the 
leading term logk/k is of order 1/k. 




Figure 4. Finite size scaling results for GUC at large k. Top Left Each curve shows the values 
of k[t*(j) — t*(j + 1)] as a function of j/k for k = 2 8 , 2 9 , . . . , 2 16 (from the farthest to the closest 
curve to 1), and was obtained by integrating the equations of motion (24) by finite differences. 
For each k, the value of a used is a^* uc (fc), determined as the value of a for which the maximum 
reached by 2c2(i)/(l — t) is 1. Top Right Data points of a^ uc (fc) versus logk/k + 2.15 jk (full 
red line). Bottom left The same data as above, plotted as {k x [t*(j) — t*(j + 1)]} x k 1 / 2 . The 
curves "collapse", showing fix) and confirming the value of v = 1/2. Bottom right By plotting 
the same curves on logarithmic scale it is easily seen that for x close to f(x) ~ x~^ with 
fj, = 1/2, corresponding to the slope of the full red line. 



6. Conclusions 

One of the main results of this paper, that is, that linear-time search heuristic are not able to 
solve instances in the clustered phase of UE-CSP problems should be interpreted with care. In 
XORSAT-like models the clustering transition coincide with the emergence of strong correlations 
between variables in solutions, while the two phenomena generally define two distinct critical 
ratios for other random decision problems [15, 16]. From an intuitive point of view it is expected 
that the performances of search heuristics are affected by correlations between variables rather 
than the clustering of solutions. Indeed, as the search algorithms investigated here do not allow 
for backtracking or corrections of wrongly assigned variables, very strong correlations between 
O(N) variables (recall that the backbone includes O(N) variables in the clustered phase) are 
likely to result in e -0 ^ probabilities of success for the algorithm. 

Extending the present work to the random Satisfiability (fc-SAT) problem would be interesting 
from this point of view, because even if the clustering and freezing transition coincide at leading 



order for k — > oo [3], their finite k values are different in this case. Moreover, in some similar 
problems (/c-COL [17] and l-in-/c-SAT [18]) it has been proven that search algorithms similar to 
the ones investigated here are efficient beyond the point where the replica-symmetry-breaking 
solution is stable. Therefore these algorithms might beat the clustering threshold in these 
problems. Note however that in these cases the transition is continuous, so that the structure 
of the clusters is expected to be very different from the one of XORSAT. 

In addition, while the Generalized Unit Clause heuristic is here shown to be optimal for the 
/c-XORSAT problem and to saturate the clustering ratio when k — > oo, it is certainly not the 
case of the fc-SAT problem. Determining a provably optimal search heuristic for this problem 
remains an open problem. 
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